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BEFORE THE BOARD OF PATENT APPEALS AND INTERFERENCES 



In re Application of Atty. Docket 

GERARD DE HAAN ET AL. PHNL 000643 

Serial No.: 09/913,670 Group Art Unit: 2613 

Filed: August 11, 2 001 Examiner: S.S. An 

ESTIMATION OF MOVEMENTS IN VIDEO IMAGES 

Commissioner for Patents 
P.O. Box 1450 

Alexandria, VA 22313-1450 
Sir: 

APPEAL BRIEF 

This is an appeal from the Examiner of Group 2613 finally- 
rejecting claims 1-5 and 7 in this application. 



(i) Real Party in Interest 

The real party in interest in this application is U.S. 
PHILIPS CORPORATION by virtue of an assignment from the inventors 
recorded on August 17, 2001, at Reel 012220, Frame 0076. 



(ii) Related Appeals and Interferences 

There are no other appeals and/or interferences related 
to this application. 
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(iii) Status of the Claims 

Claims 1-5 and 7 stand finally rejected by the Examiner. 
Claim 6 was withdrawn (based on an unelected species) . Applicants 
are appealing the final rejection of claims 1-5 and 7. 

(iv) Status of Amendments 

There was one (1) Response filed on January 20, 2005, 
after final rejection of the claims on December 28, 2004, which was 
considered by the Examiner. 

(v) Summary Of Claimed Subject Matter 

The subject invention relates to motion estimation in 
video signals in which two or more motion models are used to 
describe the displacement of image objects between a current and a 
previous video image. As described in the Substitute Specification 
on page 3, line 23 to page 4, line 13 (paragraph [0007]), since it 
is not clear, in advance, which motion model is to be assigned to 
which image area and with which parameter set, the selection 
criterion is initially used with all motion models for the same 
image areas . 

The subject invention seeks to simplify the amount of 
calculations needed to be performed. As described in the Substitute 
Specification on page 4, lines 21-24, only those parts of the image 
area in which the first video image is significantly distinguished 
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from the second video image are taken into account for determining 
the parameter sets. This is effectively illustrated in Fig. 1 and 
described in the Substitute Specification on page 12, lines 16-23 
(paragraph [0029] ) . In particular, Fig. 1 shows a video image 1 of 
a person riding a motorcycle 2 moving from left to right on a 
street 3. The background (and the street 3) is stationary, which 
the motorcycle is in motion. By comparing various areas of the 
image frame of a current image and a previous image, "interesting" 
image areas can be distinguished and are indicated by white blocks 
4 . 

The method of the present invention, as claimed in claim 
1, finds support in the Substitute Specification on page 5, line 18 
to page 6, line 9 (paragraph [0011]), on page 12, line 24 to page 
13, line 15 (paragraph [0030]), and in Fig. 2, wherein the 
selecting step is shown as step 9 in which a current video image 7 
is compared with a previous video image 6 to determine the 
interesting image areas. These image areas are processed in step 11 
to determine the parameters of the motion models. Fig. 2 also shows 
a threshold 8 being applied in step 9. This threshold 8, as claimed 
in claim 2, is used for determining interesting image areas in 
which the deviation between the current image and the previous 
image exceeds the threshold. This is described, in detail, in the 
Substitute Specification on page 12, lines 3-15 (paragraph [0028]). 
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The device of claim 5, is described in the Substitute 
Specification on page 6, line 24 to page 7, line 14 (paragraph 
[0014]), on page 13, line 16 to page 14, line 3 (paragraph [0031]), 
and in Fig. 3. In particular, an image memory 22 receives the video 
signal 20 as input to a digital image processing unit 21. Therein, 
a processor 23 performs the image processing method according to 
the invention, i.e., the current and previous images are compared 
to select significantly distinguished parts of the image frame (9 
in Fig. 2) , and parameter sets of two or more motion models are 
determined for these selected parts (11 in Fig. 2) . 

Claim 7 claims a computer program product for use in, for 
example, the processor 23 of Fig. 3, and is described in the 
Substitute Specification on page 8, lines 10-22 (paragraph [0016]). 

(vi) Grounds of Rejection to be Reviewed on Appeal 

(A) Whether the invention, as claimed in claims 1 and 4, is 
anticipated by U.S. Patent 5,450,133 to Herpel et al . , 
under 35 U.S.C. 102 (b) , 

(B) Whether the invention, as claimed in claims 2, 3 and 5, 
is unpatentable over Herpel et al . in view of 
International Patent Application No. WO 99/16251 to De 
Haan et al . , under 35 U.S.C. 103(a)- 
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(C) Whether the invention, as claimed in claim 7, is 

unpatentable over Herpel et al . in view of U.S. Patent 
5,933,535 to Lee et al . , under 35 U.S.C. 103(a). 

(vii) Argument s 

(A) The Herpel et al . patent discloses motion compensated 

adaptive vertical filtering of an image representative signal in 
which a motion parameter generator generates picture-by-picture 
motion parameters for a pair of pictures from the block-by-block 
motion parameters originating from a coder 15. In a format 
converter 14, picture signals from a line memory 13 are motion 
compensated by the picture-by-picture motion parameters and are 
coupled to a frame memory 22 which also receives picture signals 
delayed by one field. The output of the frame memory contains frame 
signals with maximum vertical local resolution. 

The subject invention concerns motion estimation in 
which, in determining the parameter sets, only those parts of the 
image frame are taken into account in which the first video image 
is significantly distinguished from the second video image. To that 
end, as claimed in claim 1, the subject invention includes 
"selecting parts of an image frame in which a first video image is 
significantly distinguished from a second video image" , and 
"determining, in the selected parts in the first and second video 
images, parameter sets of two or more motion models", and as 
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claimed in claim 4, the subject invention further includes "of the 
selected parts, those parts of the image area in which motion was 
determined in previous video image data of a sequence of video 
images, are taken into account for determining the parameter sets." 

It has been well established that "A claim is anticipated 
only if each and every element as set forth in the claim is found, 
either expressly or inherently described, in a single prior art 
reference." Verdegaal Bros. v. Union Oil Co. of California, 814 
F.2d 628, 631, 2 USPQ2d 1051, 1053 (Fed. Cir. 1987) . 

Appellants submit that Herpel et al . neither shows nor 
suggests each and every element as set forth in the claim. In 
particular, in rejecting claims 1 and 4, the Examiner paraphrases 
the wording in the claims and cites Fig. 2, elements 22 and 24, and 
col. 2, lines 51-55, as disclosing these limitations. However, 
element 22, as noted above, is a frame memory, while element 24 is 
an input to motion compensation circuit 23 which carries the 
picture-by-picture motion parameters from the motion parameter 
generator 16. 

Further, the passage in Herpel et al . cited by the 
Examiner (i.e., col. 2, lines 51-55) states: 

"A motion parameter generator 16 generates picture-by-picture 
motion parameters pn-1 for a pair of pictures sn-2, sn-1 (z, vx, 
vy) from the block-by-block motion parameters, for example, for 
zoom and pan, originating from coder 15." 
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Appellants submit that it should be clear that these 



components of Herpel et al . , nor any other components of Herpel et 
al . , perform the steps "selecting parts of an image frame in which 
a first video image is significantly distinguished from a second 
video image"/ and "determining, in the selected parts in the first 
and a second video images, parameter sets of two or more motion 
models", as claimed in claim 1, nor, as claimed in claim 4, "of the 
selected parts, those parts of the image area in which motion was 
determined in previous video image data of a sequence of video 
images, are taken into account for determining the parameter sets." 
The Examiner now states : 

"... Herpel et al discloses selecting parts of an 
image frame in which a first video image (20) is 
significantly distinguished from a second video image 
(23) , and determining, in the selected parts in the 
first and second video image parameter sets of at least 
two motion models (16) , wherein those selected parts of 
the image area in which motion was determined in 
previous video image data of a sequence of video images 
(col. 2, lines 51-55, col. 4, lines 21-51). 

"Note: The Examiner considers the first video image 
to be significantly distinguished from a second video 
image (inherency emphasized) because the second video 
image is a motion compensated image, thereby inherently 
distinguished from the first video image without any 
motion compensation. Further note that a motion between 
two video images creates a significant difference," 

From the above, it should be apparent that the Examiner 

does not understand the subject invention. In particular, the 

Examiner is looking at the first and second video images in toto, 

as opposed to parts of the image frame. In particular, when 

comparing parts of an image frame in a current image with the same 
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parts of the image frame in a previous image, even though there may 
be motion between the two images, there will nonetheless be areas 
where there is no motion (see, for example, Fig. 1 of the drawings 
and the description in the Substitute Specification at page 12, 
paragraph [0029] (described above) , in which the street and the 
background exhibit no motion between the previous and current 
images) . The subject invention seeks to reduce the complexity of 
the motion compensation by eliminating the areas where there is 
little or no difference between the previous and current images 
and, presumably, little or no motion, and only performing the 
determining of the parameter sets of two or more motion models for 
those areas which show significant differences between the previous 
and current images. It should be understood that in the prior art, 
as well as in Herpel et al . , the determining of the parameter sets 
is performed on the whole of the images, i.e., each block in a 
block-by-block without regard to whether it is, in fact, needed. 

Further, the Examiner appears to be looking at a motion 
compensated image as compared to an uncompensated image. This does 
not make any sense. The subject invention is comparing a current 
input video image with a previous input video image, both of which 
have not had motion compensation. 

The sections of Herpel et al . pointed out by the Examiner 

state : 

"A motion parameter generator 16 generates picture- 
by-picture motion parameters pn-1 for a pair of 
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pictures sn-2, Sn-1 (z, vx, vy) from the block-by-block 
motion parameters, for example, for zoom and pan, 
originating from coder 15." 

and 

"FIG. 2 shows picture format converter 14 in more 
detail. A first input 20 is connected to a frame memory 
22 and receives picture signals delayed by one field 
from the respective first field from field memory 12. A 
second input 21 is connected to a mot ion- compensation 
circuit 23 and receives line-delayed picture signals 
from the respective second field from line memory 13 . A 
third input 24 is also connected to the motion- 
compensation circuit and receives motion parameters 
from motion parameter generator 16, A fourth input 27 
receives output signals from alteration detector 17. 

"Picture signals from line memory 13 are motion- 
compensated by picture-by-picture motion parameters 
from the motion parameter generator according to known 
methods, and are coupled to frame memory 22. The lines 
in the line memory can be addressed via output 211. 
Frame signals with maximum vertical local resolution 
are now present at the output of the frame memory. 
These signals are filtered by subsequent vertical 
filter 25, one picture element after another, 
corresponding to the signal at the fourth input 2 7 in 
the frame or field, and are also coupled via output 26 
to alteration detector 17. 

"A sub-sampling stage 28 is connected downstream 
from the vertical filter, and picture signal sn can be 
acquired at output 29 of sub-sampling stage 28. 

"The picture-by-picture motion parameters may 
contain zoom and/or pan factors applicable to a 
complete picture, or image translating vectors for 
sections within the pictures. For example an image 
translating vector may be formed for picture element 
blocks of 8*8 picture elements." 

Appellants submit that it should be clear that there is 

no "selecting parts of an image frame in which a first video image 

is significantly distinguished from a second video image" and then 

""determining, in the selected parts in the first and a second 

video images, parameter sets of two or more motion models". 
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Further, with regard to claim 4, since there is no 
"determining, in the selected parts", then surely, there cannot be 
any "of the selected parts, those parts of the image area in which 
motion was determined in previous video image data of a sequence of 
video images, are taken into account for determining the parameter 
sets . " 

(B) The above arguments concerning Herpel et al . are 

incorporated herein. 

The De Haan et al . patent discloses motion estimation and 
motion-compensated interpolation, in which input fields of a video 
signal are applied to a plurality of parameter estimators for 
determining motion parameters, and in which to simplify the input 
to the parameter calculation means, the parameter vector is 
selected from a number of candidate parameter vectors as the one 
that has the minimal value of a match criterion. 

(1) Claim 2 

The subject invention, as claimed in claim 2, defines the 
selecting step of claim 1 as "dividing a current and a previous 
video image into respective pluralities of blocks", and "evaluating 
deviations between the current and the previous video image block 
by block, taking those blocks as said selected parts in which the a 
value of the deviation exceeds a predetermined threshold value" . It 
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should be note that this portion of the invention has nothing to do 
with motion compensation or motion estimation. Rather, this portion 
of the invention merely selects those parts of the video image for 
which the subsequent motion estimations is to be performed. 

Appellants submit that while De Haan et al . teaches 
motion estimation, De Haan et al . neither shows nor suggests 
"evaluating deviations between the current and the previous video 
image block by block". Rather, De Haan et al . evaluates the 
candidate parameter vectors and selects the one that has the 
minimal value of a match criterion calculated in accordance with a 
given formula. Nowhere in De Haan et al . is it shown or suggested 
that deviations between a current and previous image blocks should 
be evaluated. Rather, De Haan et al . evaluates the parameter 
vectors being generated by the parameter estimators. 

It is important to understand that the predetermined 
threshold value as claimed in claim 2 relates to the amount that an 
image area differs between the current and previous video images. 
While De Haan et al . "mentions" threshold, this threshold relates 
the value of a local match error of another adjusted motion 
parameter set determination. There is no way that the threshold 
mentioned in De Haan et al . could possibly relate to the threshold 
as claimed in claim 2 . 
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(2) Claim 3 

The present invention as claimed in claim 3, further 
limits claim 2, in that "the threshold value is based on the 
condition that the number of image areas taken into account for 
determining the parameter sets is limited to a predeterminable 
value" . 

The Examiner now states "Herpel et al does not 
particularly disclose the threshold value being based on the 
condition that the number of image area taken into account for 
determining the parameter sets is limited to a predetermined value. 
However, Herpel et al . discloses the number of image area taken 
into account (block by block) for determining the parameter sets 
(col, 2, lines 51-55), and De Haan et al teaches determining the 
parameter sets based on the threshold value. Furthermore, a 
threshold value is normally a predetermined value." 

Herpel et al . discloses generating picture-by-picture 
motion parameters from the block-by-block motion parameters (col. 
2, lines 51-55). However, there is no disclosure in Herpel et al . 
or in De Haan et al . of limiting the number of image areas to a 
predetermined number, and that the threshold value, concerning the 
deviation of the current and previous image, should be based on 
this condition. 
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(3) Claim 5 

Claim 5 is an independent claim and claims a device for 
motion estimation. The elements claimed in claim 5 include "a 
digital image memory for storing a current and a previous video 
image", "means for block-wise evaluating deviations between the 
current and the previous video image, and for selecting those 
blocks of the current and previous video images in which the value 
of the deviation exceeds a predeterminable threshold value" and 
"means for determining parameter sets of two or more motion models 
in accordance with a selection criterion based on said selected 
blocks" . 

The only part of this claim which Appellants concede is 
shown in Herpel et al . and/or De Haan et al . is the digital memory 
(Herpel et al . , Fig. 2, Ref. 22). With regard to the second 
limitation of claim 5, i.e., "means for block-wise evaluating 
deviations between the current and the previous video image, and 
for selecting those blocks of the current and previous video images 
in which the value of the deviation exceeds a predeterminable 
threshold value", the Examiner again cites Fig. 1, item 16, and 
col. 2, lines 51-55. 

' As Appellants had explained previously, element 16 of 
Herpel et al . is a motion parameter generator which "generates 
picture-by-picture motion parameters pn-1 for a pair of pictures 
sn-2, Sn-1 (z, vx, vy) from the block-by-block motion parameters. 
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for example, for zoom and pan, originating from coder 15." There is 
no disclosure or suggestion of block-wise evaluating deviations 
between a current and a previous image, and for selecting blocks in 
which the deviation exceed a predetermined threshold. 

The Examiner concedes that Herpel et al . does not 
specifically disclose this limitation, but states that De Haan et 
al . teaches this specific limitation, and references page 7, line 
7-20, and page 30, lines 8-11. 

Appellants submit that this is incredible! De Haan et al . 
on page 7, lines 7-20 is merely describing a block-based estimator, 
while on page 30, lines 8-11, claim 6 states "wherein for a given 
adjusted motion parameter set determination (PEi) , said weights are 
decreased for those image parts for which the local match error of 
another adjusted motion parameter set determination (PEl-PEn) falls 
below a given threshold." The only thing in common between this 
section of De Haan et al . and the limitation in claim 5 is the term 
''threshold" . However, the threshold in claim 5 relates to an amount 
by which a block in the current image differs from a block in the 
previous image, wherein when the deviation exceeds the threshold, 
the block is selected for the determination of the parameter sets. 
In De Haan et al . , the threshold relates to the match error of 
adjusted motion parameter set determination. 
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(C) The above arguments concerning Herpel et al . are 

incorporated herein. 

The Lee et al . patent discloses object-based video 
compression process employing arbitrarily-shaped features, in which 
a computer 20 has resident software for arguably performing motion 
estimation. However, Appellants submit that Lee et al . does not 
supply that which is missing from Herpel et al . , i.e., block-wise 
comparing the video data of the first and second video images and 
selecting those blocks exhibiting significant differences between 
the first and second video images, and computing parameter sets of 
two or more motion models and supplying motion data describing the 
displacement of image objects from the previous to the current 
image based on the selected blocks. 
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Based on the above arguments, Appellants believe that the 
subject invention is not rendered obvious by the prior art and is 
patentable thereover. Therefore, Appellants respectfully request 
that this Board reverse the decisions of the Examiner and allow 
this application to pass on to issue. 



Respectfully submitted, 

Edward ^ . GoodmasrT Reg . ZB , 613 
Attorney 
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It is hereby certified that this correspondence is being 
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COMMISSIONER OF PATENTS AND TRADEMARKS 
P.O. BOX 1450 
ALEXANDRIA, VA 22313-1450 



On /D^^ ^ 



By_^ 
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-r~^ — 
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(viii) Appendix 




CLAIMS ON APPEAL 

1. (Previously Submitted) A method of motion estimation in video 
image data, said method comprising the steps: 

selecting parts of an image frame in which a first video 
image is significantly distinguished from a second video image; and 
5 determining, in the selected parts in the first and a 

second video images, parameter sets of two or more motion models. 

2. (Previously Submitted) The method as claimed in claim 1, 
characterized in that said selecting step comprises: 

dividing a current and a previous video image into 
respective pluralities of blocks; 
5 evaluating deviations between the current and the previous 

video image block by block, taking those blocks as said selected 
parts in which a value of the deviation exceeds a predetermined 
threshold value . 

3. (Previously Submitted) The method as claimed in claim 2, 
characterized in that the threshold value is based on the condition 
that the number of image areas taken into account .for determining 
the parameter sets is limited to a predeterminable value, 
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4. (Previously Submitted) A method as claimed in claim 1, 
characterized in that of the selected parts, those parts of the 
image area in which motion was determined in previous video image 
data of a sequence of video images, are taken into account for 

5 determining the parameter sets. 

5. (Previously Submitted) A device for motion estimation in video 
image data, the device comprising: 

a digital image memory for storing a current and a 
previous video images- 
means for block-wise evaluating deviations between the 
current and the previous video image, and for selecting those 
blocks of the current and previous video images in which the value 
of the deviation exceeds a predeterminable threshold value; and 

means for determining parameter sets of two or more 
motion models in accordance with a selection criterion based on 
said selected blocks. 

6. (Withdrawn) A device for displaying video images, 
particularly a television or a monitor, comprising a digital image 
memory (22) in which video image data can be stored, and electronic 
means (21, 25) for processing the image data stored in the image 

5 memory and for displaying video images on a display device (28) , 
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the means (21) for processing the image data comprising means for 
determining parameter sets of two or more motion models in 
accordance with a selection criterion, characterized in that the 
means (21) for processing the image data further comprise means for 
10 block-wise evaluation of the deviations between the current and the 
previous video image and for selection of those blocks for use of 
the selection criterion, in which the value of the deviation 
exceeds a predeterminable threshold value. 

7. (Previously Submitted) A computer program product for motion 
estimation in video image data, said computer program product 
receiving, as input, a first and a second video image, said 
computer program product block- wise compares the video data of the 
5 first and second video images and selects those blocks exhibiting 
significant differences between the first and second video images, 
and said computer program product computes parameter sets of two or 
more motion models and supplies motion data describing the 
displacement of image objects from the previous to the current 
10 image based on the selected blocks. 
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